Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation

For extended periods of time, sequence generation models rely on beam search as the decoding algorithm. However, the performance of beam search degrades when the model is over-confident about a suboptimal prediction. In this work, we enhance beam search by performing minimum Bayes-risk (MBR) decoding for some extra steps at a later stage. In our experiments, we found that the conventional MBR reranking is only effective with a large beam size. In contrast, later-stage MBR decoding is shown to work regardless of the choice of beam size, and outperform simple MBR reranking. Additionally, we found that the computation of Bayes risks can be much faster by calculating the discrepancies on GPU in batch mode.